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A bstrnct 

Most lower speed (~ 10 Mb/s) local area networks use adaptive 
or random access protocols like Klliernet. Others at higher speed use 
demand assignment like token or slotted rings. 'These include Cam- 
bridge ring ami electronic token ring systems. In this paper we discuss 
fairness issues in representatives of such protocols. In particular we 
selected KDDI as a demand access protocol using tokens, CSMA/RN 
a random access protocol and DQDH a demand access protocol using 
reservations. Wo focus on fairness at tin* media access level, i.c., at- 
taining access or being excessively delayed when a. message is queued 
to be sent as a. function of network location. Within that framework 
we observe the essential fairness of KDDI, severe fairness problems 
in DQDIl and some problems for (-SMA/HN. We investigate several 
modifications and show their ameliorative effect. Finally we give a 
unified presentation which allows comparisons of the three protocols’ 
fairness when normalized to their capacity. 

*Tltis work was supported by (TT grant It F-8!)-()(KM)l , NASA grant NAG-1-908 and 
Sun Microsystems grant RF r>s)t;o n. 
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1 Introduction 


For the last ton years local area networks protocols have been dominated 
l>y Fthornot [*), l,ll] and various token tin/; protocols with a. nominal per- 
formance on the order of IUMh/s. In !!)!)() implementation of metropolitan 
area networks are coming on the market, with a performance on the order 
of lOOMb/s. For instance, Fl)l)l (17,1 l,‘2,U>] chipsets are now available with 
several vendors integrating them to make FI)I)I accessible with their prod- 
ucts. FDDI is a token ring protocol designed to rim at a speed of lOOMb/s. 
DQDB [10] is somewhat behind FDDI in reaching the market. It lias been 
developed as a .'100 Mb/s dual bus protocol (each bus is capable of handling 
1 50 Mb/s). 

If I >redictions made in the federal High 1 *t *fTorii ui-ii<*cr Computing Program 
[18/1] come true we will see wil.li’m I cm i years operational gigabit widearea. net- 
works wlrich will give individual users access to gigabit bandwidth while be- 
ing separated thousands of kilometers. In a way we will have expanded local 
area networks from lOMb/s lo 1 C • b/s and from few kilometers to thousands 
of kilometers. 

The data rates mentioned above are realized at the lowest levels of the 
ISO model of communications. Currently the most commonly used protocol 
suit for higher levels - TCT/IP - reduces the effective speed by a factor 
of up to 10 in Kthernet. Most performance studies have been done at the 
medium access level. Considerably loss information is available on the impact 
of overhead in layers above I be transport layer. As new standards - OSI - 
are being developed to improve inter-operability ol different computers, fears 
arc expressed that the impact might prove too costly to be useful for gigabit 
networks. No mai ler what I lie future will bring for the higher levels it is 
absolutely essential that the media access protocol level be as efficient as 
possible. In this paper we shall concentrate on performance problems at the 
media a.ccess level and what, if anything, can be done about these problems 
at that level. 

Traditional performance metrics lor these* protocols include delay metrics 
such as wait time for a message at a node, service time for a. message, total 
response time (the time a message arrives at the destination minus the time 
a message arrives at a node), and throughput metrics such as bits delivered 
versus bits offered as a function of the load and the one we wish to concentrate 
on: fairness metrics. 



Fairness in itself has many facets although in general it means that an 
entity should not have an advantage (as measured by some metric) over 
another entity. An entity may In* a node, as lor instance the node located 
physically at the end of the network should not have shorter access delays 
than a node in the middle of tin* network. An entity may be trallic of a 
specific type such as small messages versus large messages, or synchronous 
versus asynchronous. An entity may be load such as a node having to serve 
a large amount of I radio versus a node with little traffic and messages from 
cither node should have the same access delay. In most cases fairness is only a 
problem over small time intervals. Most any protocol is fair if its behavior is 
observed over a long time period and statistics are calculated over that time 
period. On the other hand a faculty member editing a. file remotely who gets 
a. M) second response time lo a key stroke will be upset, if his colleague at 
another node has a millisecond response time. 

The general concept of fairness has been an issue in media access proto- 
cols since the beginning of local area networking. In carrier sensed systems, 
persistence, binary backoff limited contention and a. host of other techniques 
have been used to improve* throughput, and control access fairness [19]. Colli- 
sion free protocols using basic bit. mapping provide another method for each 
station to reserve a. frame during the next transmission period. Since! ba- 
sic bit mapping is inherently unfair to the lower numbered nodes, IMAM, 
BRAP and other protocols wen* devised to alleviate the unfairness. [20] 

Since, in contention protocols, there is tin* probability, however, remote, 
that a node’s wait for access may be* unbounded, toke*n protocols were de- 
vised. 'Token protocols may maintain a priority system) based upon actual 
token rotation time so that when heavily loaded, the* ring will give higher pri- 
ority messages better access, hater discussions of FI)I)I will examine token 
rotation access conditions in greater detail. Refere*nce [f>] has demonstrated 
that over time* all stations on an Fl)l)l ring have* eepial access to transmit 
asynchronous frames, regardless of the* size* of allocated synchronous banel- 
with for individual stations. 

Slotted rings [21] provide control mechanism to establish access fairness. 
Each slot may contain access priority information or access to slots may be 
organized into rounds generated by a master station. Further access control 
can be baaed upon the* fact that node's must pass slots they empty to suc- 
ceeding stations, or nodes may occupy only a single* slot on the; ring at any 
one time;. In addition, some* slottc*d rings provide* eTtfiercnl slot types, channel 
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and normal, which can anil cannot hr reused by the emptying node, respec- 
tively. Unidirectional bus systems provide a number of access mechanisms 
including those based upon trains, cyclic polling and non-cyciic reservation 
schemes [21]. In the paper, we will examine tin* fairness for one particular 
reservation scheme, DQDIi, in much greater detail. 

In addition to fairness controls at the media access level, some fairness 
issues may be resolved by other means. For example, a node which needs 
guaranteed access may designate the information as synchronous, thus pro- 
viding regular interval access in an integrated network. 'Phis may result in 
considerably wasted throughput and/or increased response time in the case 
where the data, being generated by the nodi' is highly dynamic and should 
normally be delivered asynchouously. 

This leads to various concepts of fairness which arc not compatible with 
each other. When transmitting voice it is necessary that a certain amount 
of information be transmitted regularly. If necessary other nodes may need 
to be starved in order that the rate for voice* traffic be sustained. Here an 
absolute level of fairness is not the goal, but one should be able to specify a 
level of fairness to allow both synchronous trallic ami a. reasonable amount 
of asynchronous trallic. The remainder of the paper discusses how various 
protocols fare front the fairness point of view. 

In section two we describe liner representative protocols and their han- 
dling of fairness. We have selected Fl)l)l as a representative of demand 
access protocols using tokens and DQDIi as a representative of a demand 
access protocol using reservations both serving metropolitan area networks. 
The third one presented is, (\SM/\/RN, a protocol developed for gigabit 
wide area networks [H] . It is representative of random access protocols which 
normally have unbounded access delays. It is known that DQDI3 has the 
problem that wait time is a function of node location and we present several 
strategies to improve performance in this regard. Section three provides a 
summary of the results and also gives a comparison of the protocols involving 
the three major metrics: delay, throughput, and fairness. 


2 Fairness in three protocols 

Fairness problems normally are associated wil.li non-uniform load distribu- 
tions across the nodes is a network. A solution to this problem is load 


balancing which attempts lu match resources (bandwidth) with needs (of- 
fered load) at individual nodes. In order lor this lea tun; to handle sudden 
bursts, the reallocation of resources has to be done within milliseconds to be 
truly useful. One protocol which does so is DRAMA, (1-1,12,8,7], a protocol 
proposed for metropolitan areas. In none of the three protocols selected is 
load balancing fairness handled at the medium access level. We are cur- 
rently in the process of developing distributed algorithms for Fl)l)l, DQDH 
and CSMA/RN along the lines described in DRAMA, that is, each node, or 
group of nodes, keeps track of the overall network load and adjusts its own 
resources demands (bandwidth share) of the network accordingly. In future 
reports we shall describe our results in that diiertion. 

All three protocols allow for dillerent types ol trallic, i.e., both syn- 
chronous and asynchronous trallic can be sent. Control of the allocation 
of bandwidth to nodes is handled in all cases at higher level. Problems with 
KDDI in handling isochronous trallic, is'., guaranteed bandwidth at fixed 
intervals, have led to KDDI II which we do not discuss hen-. 

T|,<. problem we have studied in detail is the fairness to individual nodes 
compared to other nodes as measured by wait time and throughput even 
with a. uniform load distribution across the network. To that end vve have 
modeled all three protocols at the bit operational level. The models were 
implemented in Simscript for KDDI and DQDM and a mixture of C and 
Simscript for CSMA/RN. The details of the models and their validation can 

be found in (• r i,-t]- 

Although we have made extensive studies of each protocol, we have se- 
lected for this paper only one case which illustrates the fairness problems 
wc have found in these studies. The case selected is a 50 km long network 
with 50 equally spaced nodes. The trallic offered ranged from 0% - 100% 
for DQDR and KDDI and from 0% - 225% Tor CSMA/RN. Messages were 
5000 bits long (with 10% variation) and were Poisson distributed; load was 
uniformly distributed among all nodes. For each protocol we will give a set 
of figures depicting wait lime per message averaged over the entire network 
in n sec and throughput measured in percent of capacity as a function ol 
offered load respectively. Idr each of these two curves we will give two or 
more snapshots, taken at various levels of load, of the same metrics averaged 
for each node and displayed for enrli node*. 



2.1 FDDI 


I'M) I.) I is a. token ring protocol. In Figure* I a message! is on I, lie ring with 
the token appended to its end. Node* I) is the* next one* with a message and, 
ignoring the token rotation time* for the* moment, it will pick up the token 
and send its message* as shown in Figure* 2. Fair access and throughtpiit for 
individual nodes is ol >1 aine*el l>v means of a loken rotation time (TIFF) which 
guarantees that on the* ave*rage* e*ach node* will have* access to the ring within 
TRT. In the worst case* a. node* may have* to wait 2*TRT. This, however, does 
not guarantee* useful access to the* ring in that time* period. A node may ge*t 
the token but no time* may lx* left to send a message*. In the worst case - 
all nodes having messages e|ii<*iicel up - t he* toke*n ring protocol will give each 
node! in a. round-robin way nselul l ime to send m<*ssage*s although it may take* 
(// — I) *TRT where n is I he number ol nodes. For a. detailed discussion se*e 
[ 6 ]- _ • 

Figure 2 confirms the* essential fairness of FDDI; at 80% the variation 
from the mean wait time of the* net is about i!)% for individual nodes. At 
highe.T loads the absolute* amount ol the variation increase's significantly but 
the* percentage grows much slower. For instance, at. !)()% the* variation in 
wait time is * r ),2 1 //sec with the net average being 22, .‘KM) /iscc. As indicates! 
earlier though, synchronous bandwidth is not handled at this level and it can 
happen that with nodal distribution varying over l ime* that access delays will 
vary as a function of nodes. Secondly the* variation in TUT does not allow 
for isochronous traflie*. 'Thirdly the* establishment of an effective* TRT for a 
particular environme*ut is not a trivial issue* [5]. In this discussion we have 
ignored the fact t hat FDDI actually consists of dual counter-rotating rings 
for fault tolerant purpose's because this does not elb'ct the* issue of fairness 
presented he*r<\ 

2.2 CSMA/RN 

CSMA/H.N [2] is an extremedy simple* - he*nce* it should prove inexpensive* 
to implement - protocol w h i e h relie*s on the* fact, that a network looks quite* 
differently at high speed (> I HU Mb) than at low spe*e*el (~ 10 Mh). In the* 
latter case oidy a fraction of one message occupies the entire ring while in 
the forme*r case! several messages can be* on the ring, that is, if we assume 
uni-directional transmissions. 
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The protocol is as follows: if a node has a message to seiul it does so if 
it docs not detect another message* passing by. If the ring is husy the node' 
defers until it detects an idle ring at which linn* the node starts sending. If 
a sending node detects an incoming message whose destination is itself, the 
node continues sending while it takes the incoming message ofT the ring. If 
the incoming message* is for another node, the sending node interrupts the 
message it is sending and lets the incoming message pass by. The remainder 
of the interrupted message is then treated as a new message. 'This implies 
that a receiver may have to reassemble tin* fractured messages. Figure A 
depicts various slates tin* ring can be in. In [.\\ a detailed analysis of this 
protocol shows a. truly outstanding performance over a wide range of param- 
eter choices. For instance, with boards capable of handling one gigabit/s at 
each node, the net can act ually deliver a t hroughput, of two gigabits/s. 

Fairness in most cases is quite good for up to l r >()% of offered loaf! at 
I C I bps (see Figure . r >) Init problems do occur. At higher loads the varia- 
tion of the nod<*s’ averages is beginning to increase, in specific cases actual 
starvation of some of the nodes can occur when they are unable to achieve 
their share of throughput even at higher wait times. Which nodes exhibit 
starvation does not depend on the node’s physical location but who sends 
what message to who. Also this phenomenon is exhibited by some nodes for 
a short duration and starvation keeps drifting from node to node on the ring 
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along the course of (,im<\ No I, visible in the* graph is the information that till 
150% load no node starves, at 200% load four nodes suffer severe starvation 
sending from 106% to 170% and at 225% load nodes had throughput between 
(00% - 200%), twelve nodes had between 180% -100% and five nodes had be- 
low 180% throughput. Since the results are based upon random distribution, 
conditions will vary over different runs and over different intervals in a run. 

In Figure 0 we have run CSMA/R.N at the speed of KDDI - 100Mbps. 
As expected CSMA/RN performance has deteriorated although it is still 
better than KDDI in terms of throughput and wait time metrics. The point 
of instability lies between 150% and 175% of offered load instead of being 
beyond 200%. The starving node* phenomenon is observed at 100% offered 
load and it deteriorates at higher loads. At 200% load sixteen nodes arc' 
severely affected and the variance' of wa.il time' and throughput for nodes is 
even higher than CSMA/RN operation at l(»bps. 

Research is ongoing to ensure that no node' starves in this protocol. As 
indicated earlier, we are developing automatic load balancing algorithms at 
the media access level to solve* unfairness problems due* to non-uniform load 
distribution. As a side effect we expect that these* algorithms will also solve* 
the fairness problems for uniform load distribution. 

2.3 DQDB 

DQDM [10] use's a dual sloth'd bus to se*nel segments (messages are partitioned 
and reassembled bv the* protocol) of 512 hits. A node* use's one bus to send 
downstre'am and the other bus to send upstreams. Reservation bits arc sent 
in the! opposite direction lo alert nodes to let empty segments go by to the 
nodes making reservations. Since the system is totally symmetric we shall 
illustrate it with sending in one' direction only. 

Kadi node* has a reservation bit counter which keeps track of how many 
requests have' Imvii made* downstre'am. If a segme'tif arrives at a node the* 
segment ge*ts stamped with t he counter value*, say x, and the* counter is reset 
l.o zero. When x empty slots have* passed, this segment is sent in the next 
empty slot. 

When a segment arrives, the node attempts to sc*nel its reservation up- 
stream. DQDM use's one bit of the* 512 bit segment as the* reservation bit, 
hence the* node* has to wait till a se*gme*ut with a five* re*se*rvation bit arrives. 
Therefore, it is possible* that the* se*gme*nt has Imvii se*nt already while the 



reservation hit is si ill wail in*;. As a. consequence reservation hits ran queue 
up while segments are actually being sent. 

In Figure 7 we have* shown a particular instance of DQDIJ. A segment 
with the destination lor node M arrives at node A. Since* two nodes have 
already made reservat ions (reservation hit counter *2), the stamp is set to 
2, anil node* A will he aide to use tin* third empty segment, labeled ’a 1 , in 
th<* mean time, tin* reservation will have to wait till tin 1 old one (attempted 
reservation = I) gels reservation slot ’Id after which node A will send its 
reservation in slot labeled Y\ Thus, tin* segment will have been sent before 
a reservation was made. At this point the reservation bit counter will he up 
to a value of A. 

Under normal circmnsl ances, DQUM will order all segments in the net- 
work waiting to lx* sent il we neglect the time it takes for signals to roach 
the other node's. However, that does not apply to node's where queues of 
segments (and message's) have' hnilt up. Kach epicne will be ordered and all 
the front segments of the queues will be' ordered but elements in two differ- 
ent epieue's are not ne*re*ssarily ordeTed with regard to e'ach other. We shall 
demote' this case' as the* non -ordered one'. 

Wc have' elevelope*el models with eliir<*re*nt strate*gies for when to send the: 
reservation bit and wlie*the*r or not te> fully ordeT message's (and segments). 
The first strate'gy for se'ueling the reservation bit is as described above and 
we! shall reTer to it as the* nt/f/n ssire one*. In the* sc*cond strate'gy we elelay 
the stamping of the* Iron! segme'nt until the* reservation hit. has be'cu sent. 
We* re*fe*r to this strate'gy as the* itnn-ajiffn ssirt one*. A third strate'gy, the* 
combined one, use's a Me'rnouilli (rial to eh'cide' what strate'gy to employ for 
e'ach me'ssage* (not segment). 

We also investigated the* impact of fully oreh'ring segments across all 
(|uoues. In the orele*re'el ease', we* attach te> the state* of the reservation bit 
counter each segment as it c'i iters the* que'ue* auel re'se*t the* counter to zero. 
Whe'ii the* front, segment is stampe'd to start tin* count down, it. is from this 
held that the value for the* stamp is lake'll. 

Figure 8-12 are the* results of running the six combinations of reservation 
bit strategics and ordering across queues. Since we show these* combinations 
for oidy one set of parame*te'rs, a short note* on the impact of other param- 
eters on DQDH’s performance is appropriate*. In the* aggressive case, the 
wait time is strictly a function of how many reservation bits have come from 
downstream and how many segments arrive at upstream nodes before this 
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segment 's arrival ran lx 4 signaled to those 4 imeles. This, in turn, is a. function 
of the number of nodes whieli want, to send downstream and what their load 
is. In the non-aggressive rase, the wait t ime is more directly tied to the avail- 
ability of empty reservation bit slots ruining from I lie downstream side. If 
enough nodes, have enough messages arriving, ami air located within a short 
enough distance then it is possible to starve 4 a. node 4 because no reservations 
bit will be empty. 

For DQDIK we show tin 4 averages per node as a function of the node 
position and not just as a. collection of discrete points. There clearly is 
a functional relation between node positions and node performance. We 
remind tin* reader that the wait time in the following figures are for messages 
and not for segments. 

Figures 8 and 9 illustrate 4 the 4 impact that ordering lias on aggressive 
DQDB. At 80% the curvature of the nodal curve flips from concave to con- 
vex; in unstable situations the curvature is concave in both cases. Ordering 
has also a marked effect on the network average. Comparing Figures 9 and 10 
we can observe the effect of changing from an aggressive to a non-aggressive 
strategy. The non-aggressive strategy product's worse network averages, bet- 
ter performance for the middle 4 node's and worst* performance at the end 
nodes. This holds true for hol-h throughput and wait time. Figure II shows 
that a simple 4 uniform combination does not flatten out the curve although 
it produce's better performance lor more* node's and the number of badly af- 
fected node's is reel need. The cost for this improvement is a slightly worse 
network average. 

For completeness sake* we 4 give the 4 I wo remaining noil-ordered case's in 
Figures 12 and FI noting that their performances are worse 4 than their ordered 
counterparts. 

Table I give's a comparison e>f the 4 various e ase's for wait time* average's 
for the 4 entire net at 90%. It shoiihl be 4 noted that in all ease's the net is on 
the boundary e>f be'ing overloaded anel wait lime 4 pe*r se 4 is not a. meaningful 
figure. Since we* made the runs for the 4 same 4 length of simulation time we 4 
still can make meaningful relative comparisons. 

Figure M shows finally a flat curve or in other words a fair protocol. It is 
obtained by running a non-uniform combination strategy. At the cnduodcs 
the probability for using the 4 aggressive* strategy is .90 which is reduced 
parabolically down to . r > lor most e>f the* mieldle iiewles. 'This type' of com- 
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'type of Network Model 

Avg Wait Time 

Noil-ordered Non-aggressive 

44 4 0.06 | is 

Non -ordered Aggressive 

53 10.6 Its 

Noil-ordered ( ’omliiuatioii 

3091.56 jis 

Cully Ordered Non-aggressive 

4388.48 jis 

Cully Ordered Aggressive 

‘J843.6 l|is 

liilly Ordered ( 'omliiuatioii 

8858.031 *s 

Cully Ordered Non-uniform 
Combination 

7200.7 jis 


Table 1: Average W ail. Time per Network at !)()% Load 

Lined strategy produces the la i rest of the ones investigated although even 
this combination product's unfair situations at overload. These results are 
convincing enough to make us believe that DQDIl can be made fair at least 
to the degree KDDI is fair. The avenue we are pursuing, as with CSMA/ltN, 
is to solve the problem of automatic load balancing for non-uniform load at 
the cost of taking some bandwidth for global communication. This solution 
in combination with the approach above should enable us to make DQDH 
fair in all cases. 


3 Comparisons and conclusions 

KDDI develops random variations in performance ac ross nodes at overload 
but the variations are well within acceptable limits (about 20%). CSMA/RN 
shows an increase in variations of the performance metrics as load increases. 
The variation becomes significant (greater than 100%) at loads greater than 
150% for as much as 20% of the nodes and in extreme case's can lead to tem- 
porary starvation of nodes. The variations arc' not a functions of the location 
of a node but occur in random positions. DQDH shows unfairness at higher 




loads as a function of a node's’ position in the' network. Depending on the 
strategy selected end node's are hotter or worse' off than middle nodes. One 
particular, non-uniform combination ol an aggressive and a non-aggressive 
|)QI)I$ version produces fair performance up until overload. 

Clearly, comparing the performance of a 100 Ml>/s, a 300 Mb/s, and a 
Cl,/s protocol is akin to comparing apple's and orange's, lint. vv<' wanted to 
see how the protocols compare' in terms of lairness for a. fixed data rate. 
Therefore, Figure 13 and M give first a comparison for the network averages 
of the three protocols followed with nodal snapshots at 81) Mb/s, 2/0 Mb/s 
and 2 G/bs. When viewed in isolation Fl)l)l was the most trouble free but 
when compared with the other protocols at constant capacity it actually 
is the worst. Other factors such as reliability and cost though will change 
this picture for dilfcrent environments and no one absolute statement can be 
made. 
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Figure 15: Comparison of access delays for Fl)l)l, DQDI1, and CSMA/RN 



THROUGHPUT 
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Figure lf>: Comparison of t limiiglipul.s Idr l , ’l)l)l, IKJDM and CSMA/U.N 
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